1. Field of the Invention
This invention relates to an abnormal pattern detection processing method and system. This invention particularly relates to an improvement in processing of an image, on which an abnormal pattern detection processing is to be performed.
2. Description of the Prior Art
In medical fields, various kinds of image forming modalities (i.e., image input apparatuses), such as computed tomography (CT) scanners, magnetic resonance imaging (MRI) apparatuses, and computed radiography (CR) apparatuses, have become popular as apparatuses for forming images to be used in making a diagnosis. Also, with the rapid advances made in network techniques in recent years, there has arisen a tendency toward utilization of networks for connecting image forming modalities, which are located in examination rooms of hospitals, or the like, and terminals or image output devices, such as image display devices (cathode ray tube (CRT) display devices, liquid crystal display devices, and the like), and printers (laser printers (LP), e.g. laser imagers, and the like), which are located in consultation rooms, laboratories, and the like. In such cases, visible images to be used in making a diagnosis can be reproduced in the consultation rooms from image signals, which have been acquired with the image forming modalities located in the examination rooms.
Further, it has been proposed to provide a quality assurance workstation (QAWS) on the medical image network constituted in the manner described above. The QAWS collectively manages image signals, which have been acquired with various kinds of image forming modalities, processed image signals, which have been obtained from image processing performed on the acquired image signals such that image quality may be enhanced, and the like.
Before the acquired image signals are transferred to various kinds of terminals or the image output devices located on the network, the QAWS checks fundamental image quality of the image signals and acts as a server for storing the image signals. Specifically, before the signals are transferred to the terminals or the image output devices, the QAWS simply checks the presence or absence of severe defect of images, which may occur due to a failure in image recording performed by the image forming modalities, a mistake made during readout of the recorded images, inappropriate processing in various kinds of intermediate processing, or the like.
Examples of image signals, which are acquired with the image forming modalities and fed into the QAWS, include image signals representing mamma images. The mamma images are recorded primarily for making a diagnosis of breast cancer. The applicant proposed abnormal pattern detection processing systems (computer aided medical image diagnosing systems) for automatically detecting an abnormal pattern, which suggests the presence of breast cancer, or the like, from a mamma image, or the like, by the utilization of a computer, and reproducing and displaying the abnormal pattern in various display forms. The abnormal pattern detection processing systems are described in, for example, Japanese Unexamined Patent Publication Nos. 8(1996)-294479 and 8(1996)-287230.
As abnormal pattern detecting means for performing processing for detecting an abnormal pattern, the abnormal pattern detection processing systems are provided with abnormal pattern detecting means utilizing an iris filter, wherein image density gradients (or luminance gradients) in an image are represented by image density gradient vectors, and an image area, which is associated with a high degree of centralization of the image density gradient vectors, is detected as an abnormal pattern. Alternatively, the abnormal pattern detection processing systems are provided with abnormal pattern detecting means utilizing a morphology filter, wherein a multi-structure element in accordance with the size of an abnormal pattern to be detected is utilized, and an image area, at which the image density changes in a range spatially narrower than the multi-structure element, is detected as an abnormal pattern. With the abnormal pattern detecting means utilizing the iris filter, a tumor pattern (a form of the abnormal pattern), which is a form of breast cancer, or the like, can be detected automatically. With the abnormal pattern detecting means utilizing the morphology filter, a small calcified pattern (a form of the abnormal pattern), which is a different form of breast cancer, or the like, can be detected automatically.
However, the abnormal pattern detection processing performed by the abnormal pattern detecting means requires a large amounts of operations with respect to each of pixels constituting an image, and a heavy load is required for the operations with respect to the entire area of the image. Therefore, in the medical image network described above, in order for the load to the QAWS to be prevented from becoming heavy, the abnormal pattern detecting means should preferably be provided as special-purpose detection means independent of the QAWS.
In cases where the abnormal pattern detecting means is constituted as a device independent of the QAWS, it is necessary for the operator to perform an operation for transferring only the mamma image signals, which are among a plurality of image signals fed from all image forming modalities into the QAWS, from the QAWS into the abnormal pattern detecting means.
As for the mammography, a single plan image and a single side image are recorded for each of the right and left mammae. Therefore, ordinarily, four mamma images are recorded per patient. When a medical doctor, or the like, finally makes a diagnosis, the four mamma images are seen and compared with one another, and the position of occurrence of an abnormal pattern, the form of occurrence of the abnormal pattern, and the like, are checked. Accordingly, in the abnormal pattern detecting means, four mamma image signals representing the four mamma images of each patient should preferably be processed together. Also, the four mamma image signals representing the four mamma images of each patient should preferably be fed as a single set into the image output device. For such purposes, it is necessary for the operator to perform an operation for grouping the four mamma image signals representing the four mamma images of each patient into a single set during the classifying operation described above.
However, a large number of image signals are fed from a plurality of image forming modalities, which are connected to the network, into the QAWS. Therefore, considerable time and labor are required for the operator to perform the manual operations for extracting only the mamma image signals from the large number of image signals, grouping the image signals corresponding to each patient into a single set, and transferring each set of the image signals into the abnormal pattern detecting means. Also, in cases where the number of the image signals to be processed by the QAWS becomes very large, it is expected that the operator cannot manually cope with the large number of the image signals. Further, it often occurs that the abnormal pattern detection processing should be performed urgently when the operator is absent.
The primary object of the present invention is to provide an abnormal pattern detection processing method, wherein image signals, which are among a plurality of given image signals and which are to be subjected to abnormal pattern detection processing, such as image signals representing mamma images or chest images, are capable of being automatically grouped into a single set for each patient, and an operator need not perform manual operations for the grouping of the image signals.
Another object of the present invention is to provide a system for carrying out the abnormal pattern detection processing method.
An abnormal pattern detection processing method and system in accordance with the present invention are characterized by selecting image signals, which are among a plurality of received image signals and which are to be subjected to abnormal pattern detection processing, such as image signals representing mamma images, transferring only the selected image signals into abnormal pattern detecting means, grouping the image signals, which represent images of a single object varying in object orientation, or the like, into a single set in units of a patient, and performing abnormal pattern detection processing with the abnormal pattern detecting means.
Specifically, the present invention provides an abnormal pattern detection processing method, comprising the steps of:
i) feeding image signals from a predetermined image input apparatus into image signal selecting means, each of the image signals being provided with subsidiary information capable of specifying a kind of an object and a patient,
ii) selecting image signals, which are among the image signals having been received from the image input apparatus and which represent images of an object to be subjected to abnormal pattern detection processing performed by abnormal pattern detecting means, the selection being performed by the image signal selecting means,
iii) feeding an image signal, which represents an image of the object to be subjected to the abnormal pattern detection processing, from the image signal selecting means into input monitoring means,
iv) monitoring feeding of all of the other image signals, which are to be grouped into a single set together with the image signal representing the image of the object to be subjected to the abnormal pattern detection processing and which represent images of the same object of the same patient, from the image signal selecting means into the input monitoring means,
v) in cases where it has been detected that all of the other image signals have been fed into the input monitoring means, feeding all of the received image signals, which represent the images of the same object of the same patient and which are to be subjected to the abnormal pattern detection processing, in a single set from the input monitoring means into the abnormal pattern detecting means,
vi) performing the abnormal pattern detection processing on the image signals having been fed in the single set into the abnormal pattern detecting means, an abnormal pattern embedded in at least one of the images represented by the image signals being thereby detected, and
vii) feeding at least image signal components, which represent the abnormal pattern, into a predetermined image output device.
The present invention also provides a system for carrying out the abnormal pattern detection processing method in accordance with the present invention. Specifically, the present invention also provides an abnormal pattern detection processing system, comprising:
i) abnormal pattern detecting means for performing abnormal pattern detection processing on received image signals, detecting an abnormal pattern embedded in at least one of images represented by the image signals, and feeding at least image signal components, which represent the abnormal pattern, into a predetermined image output device,
ii) image signal selecting means for receiving image signals, each of which is provided with subsidiary information capable of specifying a kind of an object and a patient, from a predetermined image input apparatus, selecting image signals, which are among the image signals having been received from the image input apparatus and which represent images of an object to be subjected to the abnormal pattern detection processing performed by the abnormal pattern detecting means, and feeding out the selected image signals, and
iii) input monitoring means for operating such that:
a) in cases where an image signal, which represents an image of the object to be subjected to the abnormal pattern detection processing, is received from the image signal selecting means, the input monitoring means monitors feeding of all of the other image signals, which are to be grouped into a single set together with the image signal representing the image of the object to be subjected to the abnormal pattern detection processing and which represent images of the same object of the same patient, from the image signal selecting means, and
b) in cases where it has been detected that all of the other image signals have been fed into the input monitoring means, the input monitoring means feeds all of the received image signals, which represent the images of the same object of the same patient and which are to be subjected to the abnormal pattern detection processing, in a single set into the abnormal pattern detecting means.
The term xe2x80x9cabnormal pattern detecting meansxe2x80x9d as used herein means the means for automatically detecting an abnormal pattern, which suggests the presence of breast cancer, or the like, from a mamma image, a chest image, or the like. Such means is disclosed in, for example, Japanese Unexamined Patent Publication Nos. 8(1996)-294479 and 8(1996)-287230. By way of example, the abnormal pattern detecting means may be the means utilizing an iris filter, wherein an image area, which is associated with a high degree of centralization of image density gradient vectors, is detected as an abnormal pattern. Alternatively, the abnormal pattern detecting means may be the means utilizing a morphology filter, wherein an image area, at which the image density changes in a range spatially narrower than a multi-structure element, is detected as an abnormal pattern.
Also, the term xe2x80x9cfeeding at least image signal components, which represent an abnormal pattern, into a predetermined image output devicexe2x80x9d as used herein means that only the image signal components representing the detected abnormal pattern may be fed into the predetermined image output device, or that the image signal components representing the detected abnormal pattern and the original image signal representing the original image may be fed together into the predetermined image output device. In cases where the abnormal pattern detecting means feeds the image signal components representing the detected abnormal pattern and the original image signal representing the original image together into the predetermined image output device, the abnormal pattern and the original image may be reproduced and displayed automatically in accordance with a predetermined layout. Alternatively, layout conditions may be specified previously from a terminal, which is connected to the image output device, and the abnormal pattern and the original image may be reproduced and displayed in accordance with the specified layout conditions.
The image layout may take one of various known forms, such as those disclosed in Japanese Unexamined Patent Publication No. 8(1996)-294479. For example, the original image and the abnormal pattern may be reproduced and displayed as independent images in a multi-window form on a single display screen or a single output medium. Alternatively, the original image and the abnormal pattern may be superposed one upon the other and displayed in this form in a single window. As another alternative, a plurality of images (e.g., right and left mamma images), which are to be grouped into a single set, may also be reproduced and displayed in a multi-window form on a single display screen or a single output medium.
Examples of the image input apparatuses include various kinds of image forming modalities (such as CT scanners, MRI apparatuses, and CR apparatuses), and memories storing image signals. Examples of the image output devices include image display devices (CRT display devices, liquid crystal display devices, and the like), and printers (laser printers (LP), e.g. laser imagers, and the like). The image input apparatuses and the image output devices may be connected to a network, such as a medical image network.
The term xe2x80x9csubsidiary informationxe2x80x9d as used herein means image recording menu information, identification (ID) information for specifying a patient, information representing the date on which the image was recorded, a combination of the above-enumerated pieces of information, and the like. The image recording menu information represents the kind of the object (i.e., the site of a human body, such as the head, the chest, the abdominal cavity, the mamma, the neck, or the extremities), the object orientation (such as a frontal orientation or a lateral orientation), in which the object was placed when the image of the object was recorded, and the image recording technique used (such as simple image recording, tomography, or contrasted image recording).
The term xe2x80x9cimage signals representing images of an object to be subjected to abnormal pattern detection processingxe2x80x9d as used herein means the image signals representing the mamma images, the chest images, and the like. As for such an object, ordinarily, a plurality of images are recorded in a single examination with respect to a single patient.
The image signal selecting means selects the image signals, which represent the images of the object to be subjected to the abnormal pattern detection processing. The selection should preferably be performed in accordance with the subsidiary information, which is appended to each image signal and which represents the kind of the object. The image signals having been selected by the image signal selecting means are fed into the input monitoring means. The other image signals, which are not fed from the image signal selecting means into the input monitoring means, may be fed from the image signal selecting means into image processing means, the image output device, or the like.
By way of example, in cases where the image signals representing the images of an object are the mamma image signals, ordinarily, a single plan image and a single lateral image are recorded with respect to each of the right and left mammae in a single examination. Therefore, four mamma image signals per patient are grouped into a single set. Accordingly, in cases where a first mamma image signal is fed into the input monitoring means, the remaining three mamma image signals correspond to xe2x80x9call of the other image signals, which are to be grouped into a single set together with the image signal representing the image of the object to be subjected to the abnormal pattern detection processing and which represent images of the same object of the same patient.xe2x80x9d The foregoing also applies to objects other than the mammae. For example, as for chest images, an image signal representing a plan object image (also referred to as a frontal object image) and an image signal representing a lateral object image may be grouped into a single set. In such cases, when one of the image signal representing the plan object image and the image signal representing the lateral object image has been fed into the input monitoring means, the other image signal corresponds to xe2x80x9call of the other image signals, which are to be grouped into a single set together with the image signal representing the image of the object to be subjected to the abnormal pattern detection processing and which represent images of the same object of the same patient.xe2x80x9d
A judgment as to whether all of the image signals, which represent the images of the same object of the same patient, have or have not been fed into the input monitoring means may be made in the manner described below. Specifically, for example, a reference table, which specifies the number of image signals to be inputted with respect to the same object, may be prepared in accordance with the kind of the object. A judgment may be made by making reference to the reference table and in accordance with the subsidiary information appended to the inputted image signal. Also, the feeding of all image signals may be monitored in accordance with the patient ID information and the image recording menu information. Further, ordinarily, as for the same object of the same patient, only a specified number of images (varying in object orientation, image recording direction, or the like) are recorded per day. Therefore, the monitoring may be made in accordance with the patient ID information, the site in the patient the image of which was recorded (one of pieces of the image recording menu information), and the number of images recorded ordinarily.
For example, in the cases of the mammography, it often occurs that image signals representing only either one of the right and left mamma images are to be used (e.g., one of the right and left mammae has been cut off, and therefore image recording is not necessary for the cut-off side). In such cases, all of the four mamma images are not recorded. In such cases, even if the monitoring of the feeding of the four mamma image signals is continued, the completion of the feeding cannot be detected. Also, it may occur that at least one of the image signals to be grouped into a single set is not fed due to a certain cause. In such cases, if the input monitoring means waits for the feeding of all image signals to be grouped into a single set, the input monitoring means cannot appropriately perform the next process, i.e. the process for feeding all of the received image signals, which represent the images of the same object of the same patient and which are to be subjected to the abnormal pattern detection processing, in a single set into the abnormal pattern detecting means.
Therefore, in cases where the input monitoring means received a first image signal with respect to a specific patient and did not detect the feeding of all of the other image signals, which represent images of the same object as the object represented by the first image signal with respect the patient, within a predetermined length of time after the receiving of the first image signal, the input monitoring means may regard that the feeding of all of the other image signals has been detected. In this manner, the input monitoring means may feed only the received image signals, which are to be subjected to the abnormal pattern detection processing, in a single set into the abnormal pattern detecting means. In cases where the input monitoring means is constituted in the manner described above, the problems do not occur in that the image signals to be subjected to the abnormal pattern detection processing cannot be fed into the abnormal pattern detecting means.
Also, the abnormal pattern detection processing system may further comprise set image signal setting means for previously setting a number and/or definition of all of the image signals, which are to be grouped into a single set and which represent the images of the same object of the same patient, (as in the reference table described above) and
the input monitoring means monitors the feeding of the image signals from the image signal selecting means in accordance with the number and/or the definition having been set by the set image signal setting means.
For example, in the cases of the mammography, as the number and definition of all of the image signals, which are to be grouped into a single set and which represent the images of the same object of the same patient, one of (1), (2), (3), and (4) described below may be set previously by the set image signal setting means:
(1) a single image signal representing a plan image and a single image signal representing a lateral image with respect to each of the right and left mammae (a total of four image signals),
(2) a single image signal representing a plan image with respect to each of the right and left mammae (a total of two image signals),
(3) a single image signal representing a lateral image with respect to each of the right and left mammae (a total of two image signals), and
(4) a single image signal representing a plan image and a single image signal representing a lateral image with respect to either one of the right and left mammae (a total of two image signals). In this manner, the waiting for the feeding of all of the image signals to be grouped into a single set can be performed appropriately.
With the abnormal pattern detection processing method and system in accordance with the present invention, the image signal selecting means selects the image signals, which are among the image signals having been received from the predetermined image input apparatus and which represent the images of the object to be subjected to the abnormal pattern detection processing. The selection is made in accordance with the subsidiary information appended to each of the image signals. Also, the image signal selecting means feeds the selected image signals into the input monitoring means. Therefore, the operator need not perform the manual operations for selecting the image signals to be fed into the abnormal pattern detecting means and feeding the selected image signals into the abnormal pattern detecting means.
In cases where a first image signal, which represents an image of a specific object of a specific patient, is fed from the image signal selecting means into the input monitoring means, the input monitoring means monitors the feeding of all of the other image signals, which are to be grouped into a single set together with the first image signal and which represent images of the same object of the same patient, from the image signal selecting means. The monitoring is performed in accordance with the subsidiary information appended to the first image signal. In cases where it has been detected that all of the other image signals have been fed into the input monitoring means, the input monitoring means feeds all of the received image signals, which represent the images of the same object of the same patient and which are to be subjected to the abnormal pattern detection processing, in a single set into the abnormal pattern detecting means. Therefore, the operator need not perform the operations for finding the image signals, which are to be grouped into a single set for each patient, from various kinds of many image signals, which are inputted in a random order, and grouping the found image signals into a single set.
The abnormal pattern detecting means performs the abnormal pattern detection processing on the set of the image signals grouped with respect to the same object of the same patient. Information representing the results of the abnormal pattern detection processing is fed into the image output device and reproduced in one of various layout forms.
As described above, with the abnormal pattern detection processing method and system in accordance with the present invention, operations ranging from the input of image signals to the reproduction and displaying of an abnormal pattern can be performed automatically, and the operator need not perform manual operations for such purposes. Therefore, the problems can be prevented from occurring in that a mistake is made in operator""s operations due to complicated manual operations and detection of an abnormal pattern becomes late due to the mistake made in operations. Accordingly, routine diagnosis with the detected abnormal pattern can be performed.